The present invention relates to a videophone apparatus forming one terminal of a videophone system that sends and receives picture data as well as voice data over a communication line, more particularly to a videophone apparatus that modifies or replaces a live picture of the user for purposes of courtesy and privacy protection.
In recent years, with the development of communication networks and advances in data compression technology, videophone systems that can communicate (send and receive) not only voice but also pictures over the same line have gradually been spreading. Differing from conventional telephone systems that send and receive only speech, videophone systems have made it possible to converse while seeing the other party""s face.
A videophone system and a conventional videophone apparatus will be described below with reference to the drawings.
FIG. 22 is an overall block diagram showing a videophone system using conventional videophone apparatus.
In the videophone system in FIG. 22, a videophone apparatus 1 and a videophone apparatus 21 are disposed on opposite sides of a communication network 40 such as a telephone network. The sending and receiving of voice and pictures is thus carried out through the communication network 40 between videophone apparatus 1 and videophone apparatus 21. The user of videophone apparatus 1 is a speaker 2 (Mr. A); the user of videophone apparatus 21 is a speaker 22 (Mr. B). Speaker 22 (Mr. B) and the background 23 of Mr. B appear on the monitor screen 12a of videophone apparatus 1; speaker 2 (Mr. A) and the background 3 of Mr. A appear on the monitor screen 12b of videophone apparatus 21.
In videophone apparatus 1 and videophone apparatus 21 there are provided: an operation unit (OP) 11 by which the speakers 2, 22 give instructions to the videophone apparatus 1 and videophone apparatus 21; a display unit 12 that displays the faces of the far-end speakers 2, 22, their backgrounds 3, 23, and so on; an imaging unit 13 that images the faces of the speakers 2, 22, their backgrounds 3, 23, and so on; a handset 14 by which the speakers 2, 22 input and output their voices; a system control unit (SYSCON) 15 that controls the coding and decoding of voice data and picture data and the operation of the videophone apparatus 1 or 21 overall according to the instructions input from the operation unit 11; a voice-data coding/decoding unit 16 that codes signals input from the handset 14 and decodes signals sent to the handset 14; a picture-data coding/decoding unit 17 that codes signals input from the imaging unit 13 and decodes signals sent to the display unit 12; a transmission control unit (TXCON) 18 that converts signals output from the voice-data coding/decoding unit 16 and picture-data coding/decoding unit 17 to signals for transmission on the communication network 40, extracts picture data and voice data from incoming transmitted signals on the communication network 40, and outputs these data to the voice-data coding/decoding unit 16 and picture-data coding/decoding unit 17; and a line interface unit 19 that carries out adjustments of the signal level, signaling system, and so on in order to adapt the output signals of the transmission control unit 18 to the communication network 40, and carries out adjustments of the signal level, signaling system, and so on so that signals received from the communication network 40 can be processed inside the videophone apparatus 1, 21.
Next, the coding of the transmitted pictures in FIG. 22 will be explained in more detail.
FIG. 23 is a drawing that shows selected blocks related to the process of coding transmitted pictures in the videophone apparatus 1 in FIG. 22.
A line-connect/disconnect key (off-hook/on-hook key) 11a is provided in the operation unit 11 for the speaker 2 to use to give instructions for the process of connecting the videophone apparatus 1 to the line and the process of disconnecting the videophone apparatus 1 from the line.
The picture-data coding unit 90 is a block that indicates a coding unit used in sending picture data in the picture-data coding/decoding unit 17 in FIG. 22. Descriptions of the other blocks will be omitted because they duplicate blocks shown in FIG. 22.
A more detailed description of the picture-data coding unit 90 in FIG. 23 will now be given.
FIG. 24 is a block diagram showing the internal structure of the picture-data coding unit 90 in FIG. 23.
The internal structure of the picture-data coding unit 90 in FIG. 24 also shows the picture signal coding system shown, for example, in ISO-IEC/JTC1/SC29/WG11 MPEG 92/NO245 Test Model 2.
The picture-data coding unit 90 in FIG. 24 comprises a subtractor 52, a DCT (discrete cosine transform) circuit 53, a quantizing circuit 54, a variable-length coding circuit 55, a transmit buffer 56, a dequantizing circuit 61, an IDCT (inverse discrete cosine transform) circuit 62, an adder 63, a memory circuit 64, a switching circuit 65, and a motion compensation and prediction circuit 66.
On instruction from the system control unit 15, a signal 71 (a digital picture signal) input to the picture-data coding unit 90 from the imaging unit 13 is input to the first input unit (1) of the subtractor 52, the first input unit (1) of the motion compensation and prediction circuit 66, and the second input unit (2) of the quantizing circuit 54. The output 72 of the subtractor 52 is input to the DCT circuit 53; the output 73 of the DCT circuit 53 is input to the first input unit (1) of the quantizing circuit 54. The output 74 of the quantizing circuit 54 is input to the variable-length coding circuit 55 and is also input to the dequantizing circuit 61. The output 75 of the variable-length coding circuit 55 is input to the transmit buffer 56. The output from the first output unit  less than 1 greater than  of the transmit buffer 56 is output to the transmission control unit 18; the output 77 from the second output unit  less than 2 greater than  of the transmit buffer 56 is input to the third input unit (3) of the quantizing circuit 54.
The output 81 of the dequantizing circuit 61 is input to the IDCT circuit 62; the output 82 of the IDCT circuit 62 is input to the first input unit (1) of the adder 63. The output 83 of the adder 63 is input to the first input unit (1) of the memory circuit 64; the output 84 of the memory circuit 64 is input to the second input unit (2) of the motion compensation and prediction circuit 66 and the first input unit (1) of the switching circuit 65. The output from the first output unit  less than 1 greater than  of the motion compensation and prediction circuit 66 is input to the second input unit (2) of the memory circuit 64.
In addition, a zero signal is furnished to the second input unit (2) of the switching circuit 65, and the output 87 from the second output unit  less than 2 greater than  of the motion compensation and prediction circuit 66 is input to the third input unit (3) of the switching circuit 65. The output 85 of the switching circuit 65 is input to the second input unit (2) of the subtractor 52 and the second input unit (2) of the adder 63.
The videophone system and videophone apparatus shown in FIGS. 22 to 24 operate as follows.
A case will be described in which, for example, speaker 2 (Mr. A) uses videophone apparatus 1 to call speaker 22 (Mr. B), who is at videophone apparatus 21.
First, when the line-connect/disconnect key 11a in the operation unit 11 shown in FIG. 23 is operated by speaker 2, the system control unit 15 connects videophone apparatus 1 to the communication network 40 by means of the transmission control unit 18, line interface unit 19, and so on.
Next, speaker 2 designates the dial number of speaker 22 by means of dial keys or the like (not visible), and a communication line is thereby connected between videophone apparatus 1 and videophone apparatus 21.
Whereupon, a picture of the far-end speaker 22 is displayed on the monitor screen 12a in the display unit 12 of videophone apparatus 1, together with the speaker""s background 23, and a picture of speaker 2 is displayed on the monitor screen 12b in the display unit 12 of videophone apparatus 21, together with the speaker""s background 3. Mutual sending and receiving of the pictures and voices of speaker 2 and speaker 22 thus becomes possible.
A case will also be described in which, for example, speaker 2 (Mr. A), who was using videophone apparatus 1 to converse with speaker 22 (Mr. B) at videophone apparatus 21, terminates the communication.
When the line-connect/disconnect key 11a in the operation unit 11 shown in FIG. 23 is operated by speaker 2, the system control unit 15 disconnects the communication line that was established between videophone apparatus 1 and the communication network 40 by means of the transmission control unit 18, line interface unit 19, and so on.
Whereupon, the picture of the far-end speaker 22 that was being displayed together with the speaker""s background 23 on the monitor screen 12a of the display unit 12 of videophone apparatus 1 disappears, and the picture of the speaker 2 that was being displayed together with the speaker""s background 3 on the monitor screen 12b of the display unit 12 of videophone apparatus 21 disappears simultaneously. Communication between speaker 2 and speaker 22 thereby terminates.
The operation of the above-described picture-data coding unit 90 will be explained only briefly, because it is standardized by organizations such as the ISO-IEC as described above. To reduce redundancy in the time-axis direction in the input signal 71 input from the imaging unit 13, that is, the picture data imaged in videophone apparatus 1, a subtraction process is performed in the subtractor 52 to obtain the inter-picture difference with the picture data output through the motion compensation and prediction circuit 66. A DCT (discrete cosine transform) in the spatial-axis directions is performed in the DCT circuit 53 on the difference signal (output 72) output from the subtractor 52. The discrete-cosine-transformed difference signal (DCT coefficients; output 73) output from the DCT circuit 53 is quantized in the quantizing circuit 54, and after the quantized signal (output 74) has been variable-length coded in the variable-length coding circuit 55, it is output through the transmit buffer 56 to the transmission control unit 18.
As described above, the imaged picture data are coded and transmitted as a signal with a small amount of information by means of information compression by the DCT (discrete cosine transform), quantization, and variable-length coding. Moreover, the receiving side is adapted to obtain the original amount of information by decoding the information-compressed signal. In a videophone system, this enables pictorial communication over lines that conventionally carried only voice communication.
In contrast to telephone apparatus that sent and received only voice signals over a telephone line, a videophone apparatus as described above could be said to be a telephone apparatus that can also send and receive pictures, by transmitting signals coded using information-compression technology on the transmitting side, and decoding the received signals on the receiving side. Because it can also send and receive pictures, a videophone apparatus has the advantage that one can converse with a party who is far away while seeing the party""s face, but there are problems related to privacy protection that arise because the apparatus is a videophone apparatus.
For example, when the user on the sending side makes a call and the user on the receiving side picks up the handset, as soon as the line is connected, the other party""s face is displayed on the monitor screen of the videophone apparatus on both the sending side and the receiving side. Usually, however, the user on the receiving side is not expecting to be called and is not prepared, so that user""s clothing or hairstyle may be in disarray, or it may happen that the surroundings of the user on the receiving side are untidy. From the standpoint of privacy protection, it is not pleasing to the user on the receiving side to have the above sort of scene displayed on the monitor screen on the sending side.
Also, when a conversation using a videophone apparatus is placed on hold to call another person or hunt for reference materials, for example, the voice is placed on hold, but the conditions on the holding side are displayed on the monitor screen of the other party""s videophone apparatus. The actions of the user on the holding side, who may be calling to a person in the same room or an adjacent room or elsewhere, or hunting for materials, are not a meaningful display to the person watching the monitor screen of the other videophone apparatus, and they are a display of background or actions that the user on the holding side does not want to let another person see. Accordingly, from the standpoint of privacy protection, it is not pleasing to the user on the holding side to have the above sort of conditions displayed on the other party""s monitor screen.
To eliminate problems such as the above, one might consider not transmitting picture data immediately after the line is connected, for example, or during a hold, but then the monitor screen of the videophone apparatus would become similar to a screen in the non-operating (power-off) state, making the user uncertain as to whether the communication line were connected or not, and causing him or her to suspect line trouble or a malfunction of the telephone set.
Furthermore, with a videophone apparatus, when the conversation ends and the line is disconnected, if the user on one side performs an operation that disconnects the line, the face that was being displayed on the other party""s monitor screen suddenly ceases to be displayed (disappears), so the user of the videophone apparatus on the other side of the conversation receives a mild shock. Moreover, the user of the videophone apparatus on the other side of the conversation sometimes feels uneasy because he or she cannot tell whether the disappearance of the face from the monitor screen is a normal line disconnection due to the operation of the user on the other side, or is due to line trouble or a malfunction of the telephone set.
Also, even with an ordinary telephone apparatus that performs voice communication only, elderly people in particular sometimes feel that it is rude to make the other party aware that the line has been disconnected by being the first to hang up, so the party who hangs up first may be left with bad feelings. With a videophone apparatus, since not only the other party""s voice but also the other party""s face suddenly vanishes due to line disconnection, the act of hanging up first (disconnecting the line first) may be interpreted as even more rude then with a voice-only telephone. The user of a videophone apparatus who first disconnects the communication line may accordingly be left with extremely bad feelings, depending on that user""s sense of customs and manners.
The present invention addresses the above problems.
An object of the present invention is to provide a videophone apparatus that can make the far-end videophone apparatus user aware that the line is connected by sending a processed picture instead of a live picture, in order to protect the privacy of the user on the picture-sending side at the beginning of a call, for example, or when a call is placed on hold.
Another object is to provide a videophone apparatus with which the videophone apparatus user can terminate a call with an easy conscience and without feeling badly afterwards, by having the face of the other party that was being displayed on the monitor screen disappear gradually when the line to the other party""s videophone apparatus is disconnected.
The invented method of sending pictures from a first videophone apparatus to a second videophone apparatus during a call placed between the first videophone apparatus and the second videophone apparatus includes the steps of:
taking a live picture of the user of the first videophone apparatus;
sending the live picture to the second videophone apparatus during one part of the call;
generating a processed picture different from the live picture; and
sending the processed picture to the second videophone apparatus during another part of the call.
In one aspect of the invention, the processed picture is prepared in advance and stored in a memory unit of the first videophone apparatus.
In another aspect of the invention, the processed picture is obtained by combining the live picture with a prepared picture stored in a memory unit in the first videophone apparatus. For example, part of the live picture may be combined with part of the prepared picture, the two parts being displayed separately in the processed picture. Alternatively, the live picture may be overlaid semi-transparently on the prepared picture. The combining ratio, e.g., the size ratio for the two parts, or the transparency ratio of the live picture, may be varied at, for example, the beginning of the call, the end of the call, the beginning of a period in which the call is placed on hold, or the end of a period in which the called is placed on hold.
In another aspect of the invention, the processed picture is obtained by increasing a quantization coefficient of the live picture, thereby producing a mosaic effect that disguises the live picture. The quantization coefficient, hence the cell size of the mosaic effect, may be varied at, for example, the beginning of the call, the end of the call, the beginning of a period in which the call is placed on hold, or the end of a period in which the call is placed on hold.
The invention also provides a videophone apparatus having an imaging unit that takes a live picture of a user and outputs live picture data, a picture-data coding/decoding unit that codes the live picture data for transmission on a communication line and decodes picture data received from the communication line, a display unit that displays the decoded picture data, and an operation unit by which the user enters instructions. The videophone apparatus additionally comprises a picture-data memory unit storing prepared picture data, a picture-data processing unit that combines the live picture data with the prepared picture data to generate combined picture data, and a system control unit that causes the combined picture data to be coded in place of the live picture data for transmission on the communication line, responsive to an instruction entered by the user.
The invention also provides another videophone apparatus having an imaging unit that takes a live picture of a user and outputs live picture data, a picture-data coding/decoding unit that codes the live picture data for transmission on a communication line and decodes picture data received from the communication line, a display unit that displays the decoded picture data, and an operation unit by which the user enters instructions. This videophone apparatus additionally comprises a quantization coefficient control unit that controls a quantization coefficient used by the picture-data coding/decoding unit in coding the live picture data, and a system control unit that causes the quantization coefficient control unit to alter the quantization coefficient responsive to an instruction entered by the user.